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Abstract —In this paper, a modified algorithm for the detection 
of nasal and temporal eye corners is presented. The algorithm is a 
modification of the Santos and Proenka Method. In the first step, 
we detect the face and the eyes using classifiers based on Haar- 
like features. We then segment out the sclera, from the detected 
eye region. From the segmented sclera, we segment out an 
approximate eyelid contour. Eye corner candidates are obtained 
using Harris and Stephens corner detector. We introduce a post- 
pruning of the Eye corner candidates to locate the eye corners, 
finally. The algorithm has been tested on Yale, JAFFE databases 
as well as our created database. 

Index Terms —Eye corner detection; Haar-like features; 
CLAHE; Harris and Stephens 

1. INTRODUCTION 

Eye corners are the regions where the upper and lower 
eyelids meet m The two comers are technically named as 
temporal and nasal canthus as shown in Fig.|^ In applications, 
where eye movements are analyzed, eye comers serve as 
the reference points O. The reason behind this is that the 
eye corners are more stable than the iris since its shape or 
orientation is not affected by the gaze direction or the state of 
eye closure. 

Eye Corner detection has been a challenging problem in 
computer vision owing to the following issues. 

• eye comer in an image is not necessarily a single pixel 

• nasal eye corners can be occluded by nose 

• illumination effects and shadows may remove information 
related to the comer 


TABLE I 
Earlier Works 


Author 

Work 

Xia et al. (3 

Variance Projection Functions 

Zhu et al. [I] 

Contour extraction and ellipse 
fitting 

Xu et al. y 

Semantic Features extraction 

Xu et al. (5] 

Improved Local Projection 
Functions and Circle Integral 

Santos and Proenca (3 

Eye contour approximation on 
sclera segmented eye image 







Eye Detection 


Candidate 


Sclera 

comer detection 


segmentation 


The above issues make eye comer detection, a more challeng¬ 
ing task than a classical corner detection in computer vision. 
The important works which aim at addressing the eye corner 
detection issue are provided in Table |T| 

This work attempts to improve the Santos and Proenca Method 
a for eye comer detection, thereby addressing some of the 
above issues. The significant contributions of this work are as 
follows: 

• An online framework for eye comer detection 

• Improvement of the Santos and Proenka Method for 
webcam quality images 

• Post pmning of eye corner candidates 

II. Santos and Proenca Method for Eye Corner 
Detection 

In the Santos and Proenca Method a, an eye image is 
used as an input to the detector. First, they obtain a noise- 
free iris binary segmentation mask from the eye image. The 


Fig. 1. Proposed Overall Framework 



Fig. 3. Face and Eye Detection 


























next step is segmentation of the sclera region. The sclera 
being the most unsaturated region in the eye image, an HSV 
transformation yields the lowest magnitudes on the saturation 
plane. The iris and sclera being segmented out, the next stage 
constitutes of the approximation of the eyelids contour. This is 
achieved using morphological dilation of the iris segmentation 
mask with a horizontal structuring element is carried out. 
This expands the iris regions horizontally. Finally, point-by- 
point multiplication between the dilated and the enhanced 
data provides a good approximation to the eyelids contour. 
The subsequent step comprises of the generation of a set of 
candidate eye corner positions, which was performed using 
the Harris and Stephens corner detector. 

This method works well for an image having proper resolution 
and clarity. However, for inferior quality images, such as those 
obtained using a standard webcam, the performance of such 
methods is limited. 


Sclera 



Fig. 5. Temporal and Nasal Canthus 


HI. Proposed Algorithm 

Our proposed scheme is a real-time camera-based system 
depicted as a schematic in Fig. For image acquisition, the 
scheme uses a standard webcam of resolution 640 x 480 pixels. 
The algorithm has been implemented in real-time using a Sony 
PS3 Eye webcam at 30 fps. First, face detection is carried out 
in a given frame, followed by eye detection. Eye detection 
is confined to a Region of Interest (ROI) in the detected face 
area. The Rol is selected based on our previous work Q. This 
step finds each eye separately. The algorithm has been made 
robust using some preprocessing techniques for illumination 
correction. 


undergoes histogram equalization. This confines the histogram 
to a small region. An issue of this method is the amplification 
of noise if present. This issue is avoided by contrast limiting. 
After the CLAHE operation, bi-linear interpolation is applied 
to remove artifacts in tile borders. 


B. Face and Eye Detection 

The face and eye detection forms the first step in the 
algorithm for the localization of the eye comers. A classifier 
based on Haar-like features was selected for this stage. For 
optimal use, we have used parameters based on the earlier 
work m . Once the face is detected, an ROI is selected from the 
facial region. The ROI selection scheme has been reported in 
the previous work fa. Now, the search for eyes is confined to 
a reduced area, which also reduces computation and improves 
the speed. 


C. Eye Corner Detection 

The corner detection operates on the detected eye region. 

1) Sclera Segmentation: As proposed in O, the sclera is 
segmented out by converting the RGB image to HSV and 
subsequently thresholding the saturation plane. For grayscale 
images, the color space conversion is not required, and the 
thresholding operation can be applied directly. The idea behind 
this lies in the fact that the sclera is the most unsaturated 
segment in the eye image. There remain certain noisy pixels 
because of the blood vessels in the sclera. This issue is ad¬ 
dressed using the morphological opening of the sclera portion 
using an elliptical mask. 

2) Eyelid Contour Approximation: The sclera region is 
hence segmented out with the morphological post-processing. 
The boundary of the mask is overlaid on the eye image to 
obtain the eyelid contour approximation. 

3) Eye Corner Candidate selection: The corner score is 
obtained by the sum of squared differences (SSD), S{x,y) 
between the corresponding pixels of two patches in the eyelid 
contour image /(x,y). The differential of the comer score is 
considered for finding out the corner candidates. A circular 
window w(w,v) is used, to make the response isotropic. 

Six,y) =^^H'(M,v)(/(M+x,v + y)-/(M,v))^ (1) 

U V 


With Taylor series expansion and proper approximation, we 
have 


S{x,y) =[x y]H 


( 2 ) 


A. Image Enhancement 

Preprocessing of the image is necessary for filtering out 
the noise and preserving relevant information. The filtering is 
particularly essential as the eye corner is sometimes not visible 
clearly because of illumination conditions. The pre-processing 
begins by equalizing the histogram of the image. We have used 
Contrast Limited Adaptive Histogram Equalization (CLAHE) 
technique Q for this purpose. In this method, the image is 
fractioned into small blocks of size 8x8. Each of these blocks 


The Harris matrix, H is obtained as 


U V 


r f 

Ixly 



(3) 


4) Post Pruning: Since the actual eye comers will lie at the 
extreme ends of the eyelid contour, the eye corner pair having 
the farthest distance are selected as the eye corners. In cases of 
a tie, the mean comer is chosen as the correct comer estimate, 
among the corner candidates bearing equal distances. 






TABLE II 

Percentage Error in eye corner localization 



Fig. 6. Detection Results for images taken (a) Yale Database (b) JAFFE 
Database 



Fig. 7. Sample images from our created database 


IV. Results 

A face database of 30 subjects was prepared using a 
standard Sony PS3 web camera for testing the algorithm. 
Some sample images of the database is shown in Fig. The 
algorithm has also been tested on Yale Face Database ca 
and JAFFE Database lITll . A sample set of 200 images total 
were randomly selected from the databases. The percentage of 
mean-squared pixel error in eye localization has been provided 
in Table [I^ for the different databases. The error is eye corner 
localization is based on the manual marking of the end¬ 
points. The processing-rate of the algorithm with online testing 
was found to be 16.2 fps while on offline database, it was 
20.4 fps. The speed may be improved by using GPU based 
implementations, and employing parallel schemes. 

V. Conclusion 

In this paper, we propose an algorithm that uses a standard 
web camera to localize effectively the eye corner. This is 
an improvisation over the Santos and Proenca Method. The 


Database 

% Squared pixel error 

JAFFE database 

4.5 

Yale Face database 

6.5 

IITKGP database 

8.9 


significant modification lies in the post-pruning the eye comer 
candidates. The method has less than 10% localization errors 
in all the three tested databases. A future scope of the work 
may be testing of the algorithm of infrared and near infra-red 
images ca, and make appropriate modifications so that the 
applications of the algorithm can be extended to areas where 
night vision is preferable. 
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Fig. 2. The overall scheme 
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Fig. 4. Eye corner Detection 

















